Discussion of “ Least Angle Regression ” by Efron

I would like to begin by congratulating the authors (referred to below as EHJT) for their interesting paper in which they propose a new variable selection method (LARS) for building linear models and show how their new method relates to other methods that have been proposed recently. I found the paper to be very stimulating and found the additional insight that it provides about the Lasso technique to be of particular interest. My comments center around the question of how we can select linear models that conform with the marginality principle [Nelder (1977, 1994) and McCullagh and Nelder (1989)]; that is, the response surface is invariant under scaling and translation of the explanatory variables in the model. Recently one of my interests was to explore whether the Lasso technique or the nonnegative garrote [Breiman (1995)] could be modified such that it incorporates the marginality principle. However, it does not seem to be a trivial matter to change the criteria that these techniques minimize in such a way that the marginality principle is incorporated in a satisfactory manner. On the other hand, it seems to be straightforward to modify the LARS technique to incorporate this principle. In their paper, EHJT address this issue somewhat in passing when they suggest toward the end of Section 3 that one first fit main effects only and interactions in a second step to control the order in which variables are allowed to enter the model. However, such a two-step procedure may have a somewhat less than optimal behavior as the following, admittedly artificial, example shows. Assume we have a vector of explanatory variables X = (X1,X2, . . . ,X10) where the components are independent of each other and Xi, i= 1, . . . ,10, follows a uniform distribution on [0,1]. Take as model